[ Cover ]

Archived Information National A w a r d s P r o g r a m for Effective T e a c h e r Preparation Application Package Applications Due: July 3, 2000 Office of Educational Research and Improvement U.S. Department of Education OMB Approval No. 1850–0762 (Exp. 3/31/03) 1 Paperwork Burden Statement According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this information collection is 1850–0762. The time required to complete this information collection is estimated to average 50 hours per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have any comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Education, Washington, D.C. 20202–4651. If you have comments or concerns regarding the status of your individual submission of this form, write directly to: Sharon Horn, Office of Educational Research and Improvement, U.S. Department of Education, 555 New Jersey Avenue, NW, Room 506e, Washington, D.C. 20208–5644. 2 Dear Applicant: April 1, 2000 Few would debate that high quality teachers are critical to the ability of children in our nation’s schools to achieve high standards. As the student population becomes more diverse and the emphasis on academic achievement continues, the need for high quality teachers will only increase in the coming decade. Clearly, the time is right to draw attention to those teacher preparation programs that are particularly effective in preparing teachers who, in turn, are effective in helping students improve their learning. To highlight the correlation between student learning and the quality of the programs that prepare teachers, and to continue its efforts to honor excellence in education, the U.S. Department of Education is pleased to announce this first competition under the National Awards Program for Effective Teacher Preparation. To support efforts across the country to improve students’ learning in reading and mathematics, the initial year of these awards will focus on programs that prepare elementary teachers or middle and/or high school mathematics teachers. The goal of this program is to recognize teacher preparation programs whose graduates are effective in helping all K–12 students improve their learning in reading and/or mathematics. Demonstrating the link between teacher preparation programs and graduates’ ability to improve student learning is not an easy task. This awards program will recognize no more than five programs that are on the leading edge of demonstrating this link. These programs will serve as examples that will help others understand how to establish exemplary programs and gather credible evidence of effectiveness. For purposes of these awards, a “teacher preparation program” refers to a defined set of experiences that, taken as a whole, prepares participants for initial certification to teach. All programs that prepare elementary teachers or middle and/or high school mathematics teachers for initial 3 certification, including those that are not university/college-based, are eligible for this award. (Since the focus is on initial preparation, alternative certification programs are eligible while inservice programs are not.) Detailed instructions for applying for this award are provided within this application package. To be considered for the award, programs must provide compelling evidence that their graduates positively affect all students’ learning in reading and mathematics at the elementary level, or mathematics at the middle and/or high school level. Programs also must demonstrate that their graduates have a depth of content knowledge (mathematics and reading for elementary programs, mathematics for middle and/or high school programs), acquire general and content-specific pedagogical knowledge and skills, and develop skills to examine attitudes and beliefs about learners and the teaching profession. Applicants should be sure to: • respond in the suggested order to each section of the Application Narrative portion of the application; • consider carefully the guiding questions in each section; and • comply with formatting instructions. Applicants should pay particular attention to the Evidence of Effectiveness section (Section C) of the Application Narrative portion of the application. This section requires applicants to provide three types of evidence: • Formative Evidence This type of evidence demonstrates that the program gathers and uses data to make adjustments to the various stages of the program. • Summative Evidence This type of evidence demonstrates the effectiveness of the overall program in helping graduates acquire the knowledge and skills needed to improve all students’ learning. 4 • Confirming Evidence This type of evidence demonstrates the effectiveness of program graduates in K–12 settings. As mentioned previously, the goal of the awards program is to recognize teacher preparation programs that have compelling evidence of their effectiveness; consequently, all three types of evidence are required. In assembling their evidence, applicants may determine that some of the data collected to comply with Section 207 of Title II of the 1998 Amendments to the Higher Education Act of 1965 may be useful as one of many sources of evidence for this application. Please see the information about evidence in Section C of the Application Narrative portion of the application package for more details. For further information about the application or the awards program, please contact Sharon Horn at the Office of Educational Research and Improvement, U.S. Department of Education, 555 New Jersey Avenue NW, Room 506e, Washington DC 20208–5644, phone 202–219–2203, FAX 202– 219–2198, or email sharon_horn@ed.gov. Thank you for your interest in the National Awards Program for Effective Teacher Preparation. Your willingness to share information about your program will positively affect the quality of teacher preparation and, ultimately, the quality of K–12 education in the United States. Sincerely, Terry Knecht Dozier Senior Advisor on Teaching 5 6 Overview As part of a continuing effort to honor excellence in education, the U.S. Department of Education announces the National Awards Program for Effective Teacher Preparation. This program recognizes exemplary teacher preparation programs that provide evidence that their graduates are effective in helping all students improve their learning. For purposes of these awards, “all students” means all students that a teacher might encounter in a classroom (including students in regular education, students in special education, students from diverse backgrounds, and students with limited English proficiency). In keeping with the Nation’s and the Department’s priorities on reading and mathematics, this competition focuses on programs that prepare elementary teachers (since elementary teachers are typically required to teach both reading and mathematics) and on programs that prepare middle and/or high school mathematics teachers for initial certification. . Only those programs are eligible to apply. Recognition under this awards program is based on how well applicants demonstrate evidence of effectiveness. Three types of evidence must be provided: formative, summative, and confirming. The evidence must demonstrate that the program gathers and uses data to make adjustments to respond to participant needs throughout the program, ensures that participants acquire the knowledge and skills needed to improve all students’ learning by completion of the program, and gathers and uses data about its graduates’ effectiveness in K–12 settings after completion of the program. 7 The Review Process Applications will be reviewed in five stages. Stage 1: During the first stage, applications will be reviewed by U.S. Department of Education staff to determine whether they are submitted by eligible applicants, contain all necessary information (including the three types of evidence required), and meet formatting requirements. Applications will be accepted only from programs that prepare elementary teachers or middle and/or high school mathematics teachers for initial certification. Stage 2: The second stage of review, to determine semi-finalists, will be conducted by non-Departmental teams representing a broad range of teacher educators, practitioners (i.e., mathematicians, mathematics educators, K–12 teachers, and reading specialists), and policymakers (i.e., superintendents, school board members, and principals). Each application will have multiple readers and will be evaluated according to how well the application demonstrates evidence of effectiveness as described in the application package. Stage 3: In the third stage, site visits will be conducted to verify information presented in the semi-finalists’ applications and to examine additional data related to the evidence of effectiveness. Stage 4: In stage four, a non-Departmental national awards panel (different reviewers from those involved in Stage 2) will review semi-finalist applications and site visit reports, then recommend finalists to the Department. Stage 5: In the final stage, the Department will review data collected throughout the review process and select no more than five programs for national recognition. 8 Guidelines for Completing the Application General Information The purpose of the National Awards Program for Effective Teacher Preparation is to promote excellence in teaching and teacher preparation. Applicants must adequately document three types of evidence (formative, summative, and confirming) that, taken together, demonstrate their graduates are able to help all K–12 students (including students in regular education, students in special education, students from diverse backgrounds, and students with limited English proficiency) improve their learning. All three types of evidence are necessary for an application to proceed beyond the first stage of review. Applications that advance beyond stage two of the review process (i.e., those of the highest quality) will be those that provide compelling evidence from multiple indicators or measures collected for a period of two or more years. Applicants who become semi-finalists will be expected to have documentation that verifies the effectiveness of their program available for site visit reviewers. Such documentation should enable reviewers to understand the nature and quality of the program’s elements and how they contribute to graduates’ abilities to positively affect K–12 student learning. Documentation may include artifacts such as schedules, portfolios, student or teacher reflection logs or journals, assessment data, video and audio-tapes, internal and external communications, and other documents. If you have questions about the application or the awards program, contact Sharon Horn, Office of Educational Research and Improvement, 555 New Jersey Avenue NW, Room 506e, Washington, DC 20208–5644. She may be reached by phone (202–219–2203) or email (sharon_horn@ed.gov). 9 Components of the Application The application consists of the following components: Contact Information: (form provided): List the name, address, phone, fax, and email (if available) of a person with whom the Department can communicate about the application throughout the review process, including the site visit.* This person should be someone who is knowledgeable about the application. Also, list the members of the team who prepared the application and the name and address of the program. Abstract of Program: Provide a one-page abstract (see Formatting Information) that describes the characteristics that make your program exemplary, and highlights the evidence that your teacher preparation program is effective in preparing graduates who are able to help all students improve their learning. Application Narrative: Provide a narrative and accompanying materials that focus on the areas described in the Application Narrative section of the application package. * If the contact information is not applicable during the summer months, please provide a phone number where the contact person, or an alternate, can be reached during July and August. 10 Formatting Information • Attach the single-sided, one-page abstract and no more than 30 pages total of application narrative, including accompanying materials, to the contact information form provided in this packet. Any pages that exceed the equivalent of the 30-page limit will be discarded. • Margins must be at least one inch on all sides. • Type must be double-spaced and in a type size that enhances readability (at least 12 point). • Submit an original and five (5) copies of your application to Sharon Horn, Office of Educational Research and Improvement, 555 New Jersey Avenue NW, Room 506e, Washington, DC 20208–5644. Please do not bind the original. The completed application must be received no later than close of business (5:00 p.m. Eastern Time) on July 3, 2000. 11 Application Narrative Each of the four sections of the application includes two or more questions that are designed to help applicants formulate their responses. Sections A, B, and D provide reviewers with information about the context of the teacher preparation program and its potential as an example for others. Reviewers will use the information in the three sections to determine the extent to which there is a logical connection between the various aspects of the program and the results achieved. In other words, they will check for consistency between the information provided in these sections and the claims of effectiveness. Section C is the most important section of the application. In this section, applicants provide formative, summative, and confirming evidence that their program is effective in preparing graduates who are able to help all K–12 students improve their learning in reading and mathematics at the elementary level or mathematics at the middle and/or high school level. Applicants should respond to each section in the suggested order, making certain to consider carefully the guiding questions and to follow the formatting requirements described previously. Accompanying materials may be included as part of the application narrative, however, the total number of pages must not exceed 30. A. Background and Program Description In this section, applicants must provide their (1) mission statement, (2) goals and objectives, and (3) program components. If accompanying materials are used to provide information, please clearly indicate on the materials what they are intended to address and refer to the materials in the narrative by providing a list or brief description. (Please note that accompanying materials are included in the 30 page limitation.) 12 In addition to the three elements mentioned above, applicants also may consider including information about the following: • Recruitment policies for faculty and candidates • Selection procedures for faculty and candidates • Program structure (e.g., course and field experiences, support for preservice and novice teachers, mechanisms for monitoring partici pants’ progress) • State and/or district policies or mandates that affect the components of the teacher preparation program • Resources that support the program • Collaboration between the program and K–12 schools • Graduation or completion criteria In responding to this section the applicants should carefully consider the following questions: • What are the mission, goals and objectives, and components of the teacher preparation program? • How do the mission, goals and objectives, and program components relate to the effective preparation of elementary teachers or middle and/or high school mathematics teachers? B. Program’s Criteria for Effectiveness In this section, applicants must describe the principles, standards, or other criteria that the applicant uses to judge the effectiveness of its teacher preparation program. (Note: Applications are not being evaluated against a given set of principles for all programs, but are expected to include relevant criteria for guiding program improvement and modifications). 13 In responding to this section, applicants should carefully consider the following questions: • What are the criteria or standards (e.g., NCATE, INTASC, NBPTS, NCTM, state teacher licensure requirements and other appropriate standards) the program uses to evaluate its effectiveness? • How does the program ensure that program components, such as courses and instructional practices, are linked to the evaluation criteria or standards? C. Evidence of Effectiveness The response to this section must include three types of evidence: formative, summative, and confirming. Formative evidence refers to the use of data to make adjustments to the program throughout its various stages. These data are collected as participants (preservice teachers) move through the program. Summative evidence demonstrates that the program is effective in helping graduates acquire the necessary knowledge and skills to improve student learning. Such evidence is collected as preservice teachers complete the program. Confirming evidence links teacher preparation and K–12 student learning by demonstrating that program graduates are effective in helping all K–12 students improve their learning in reading and mathematics at the elementary level or mathematics at the middle and/or high school level. This evidence is collected on graduates who are employed by schools or districts. Please see the “Application Review Tool” section of the application packet for information about the rubric that will be used to review applications. The section also includes information about credibility of sources of evidence. In responding to this section, applicants must supply a brief description for each piece of supporting evidence. The description must include information about (1) the nature of the data, (2) methods used to collect the data, and (3) a summary of the data analysis. 14 Applicants must consider carefully the following questions: • What evidence is there that the program, described in section A, gathers data about the effectiveness of the various stages of the program and uses that data to make improvements? (Formative evidence) What evidence is there that the program is effective in helping graduates acquire the knowledge and skills needed to improve student learning in reading and mathematics for all elementary school students or in mathematics for all middle and/or high school students? (Summative evidence) • (Note: Evidence in this section should address graduates’ content knowledge, pedagogical knowledge and skills, and skills to examine beliefs about learners and teaching as a profession.) • What evidence is there that the program’s graduates are effective in helping all students improve their learning in reading and mathematics at the elementary school level or mathematics at the middle and/or high school level? (Confirming evidence) (Note: If there are obstacles that affect data collection (e.g., local or state regulations that prohibit the release of student data), applicants may describe these factors, and explain how they have overcome any obstacles in collecting data for purposes of evaluating the effectiveness of their program.) D. Implications for the Field A primary goal of this awards program is to share with the public, effective examples that might be adopted or otherwise used by others to improve teacher preparation programs throughout the country. In this section, applicants must discuss the challenges they have faced and overcome in administering their teacher preparation program, as well as the resulting lessons they have learned. 15 In responding to this section, applicants should consider carefully the following: • What is at least one significant challenge that the program encountered within the last five years and how was it overcome? (Note: Since demonstrating the link between teacher preparation, and K–12 student learning is a primary focus of the award program, applicants should consider describing challenges related to this issue.) • What lessons have been learned about designing, implementing, or evaluating a program that prepares graduates who are effective in helping improve student learning for all K-12 students, and how might these lessons benefit others? • What program materials (e.g., videos, Web sites, course outlines, manuals, strategies, processes) are available that could benefit others? • How have or could you help others adapt the aspects of your program that contribute most to graduates’ effectiveness with K–12 students? 16 Application Review Tool Reviewers will evaluate the information provided in each application based on three criteria: rigor, sufficiency, and consistency. These criteria, and the performance levels applicable to each, are identified in the rubric shown in Figure 1. Reviewers will use this rubric as the review instrument to judge the quality of each application. The Evidence of Effectiveness provided by an applicant under section C, the most critical portion of the application, will be evaluated on the basis of its rigor and sufficiency. The level of “rigor” applied to the evidence submitted will be determined by the extent to which the qualitative or quantitative data presented are found to be valid and reliable. The level of “sufficiency” applied to the evidence submitted will be determined by the adequacy and the extent of the data provided. The application as a whole will be evaluated on the basis of its consistency. The level of “consistency” of the application will be based on the extent to which there is a logical link between various aspects of the program as described in Sections A, B, and D of the application and the evidence of effectiveness provided under Section C. For example, if an applicant indicates in sections A, B, or D of its application that field experiences are important to the preparation of teachers, then the application should describe the variety of field experiences that are spread over the duration of the program and also include, for purposes of “consistency,” documentation of the effectiveness of these experiences. The rubric in Figure 1 identifies a range of performance levels, from 1 to 4, that reviewers will use to judge the quality of an application with regard to the three criteria—rigor, sufficiency, and consistency—explained above. Reviewers will assign a level of the rubric, 1 to 4, for each criteria based on their judgment of how well the information provided in the application matches the descriptions in the rubric of the relevant performance levels. Prior to reviewing applications, reviewers will receive training in using the rubric to ensure inter-rater reliability. 17 Figure 1. Rubric for Evaluating Evidence of Effectiveness RIGOR The evidence is highly credible. The data are valid and indicators are free of bias. Reliability is supported by multi-year data from several sources. SUFFICIENCY There are extensive data that support claims of effectiveness. The evidence includes data from multiple sources with multiple indicators. CONSISTENCY Components of the program are consistent with the vision of the program. Program components are monitored to determine if they are being instituted as designed. Evidence supports an intended, logical link between program components and program success. The consistencies support the credibility of the evidence. There are minor inconsistencies between the vision of the program and program components. Some components of program may not be monitored or there may be some inconsistencies between the evidence provided and the identified successful components of the program. The inconsistencies do not weaken the credibility of the evidence. There are several inconsistencies between the vision of the program and program components. There are significant inconsistencies between the evidence provided and the identified successful components of the program. The inconsistencies raise questions about the credibility of the evidence. There are numerous inconsistencies between the vision of the program and its components. The evidence provided is not linked to the components of the program that have been identified as contributing to the program’s success. The inconsistencies raise significant questions about the credibility of the evidence. 4 3 The evidence is credible. Validity has been addressed for most of the data. There may be some questions of bias. Reliability is supported by two or more years of data from at least one data source. There are adequate data to support the claims of effectiveness. There are multiple sources of evidence and multiple indicators for at least one source. 2 The evidence has limited credibility. The rigor is compromised by issues of bias or validity/reliability. There are no multi-year data from any source. There are limited data to support the claims of effectiveness. The data are collected from only one or two sources. There are no multiple indicators for the data source(s). 1 The evidence has little or no credibility. The rigor is significantly compromised by issues of bias, or there is not enough information to determine rigor. The data lack validity/reliability. There are no multi-year data. There are not enough data to support claims of effectiveness. There is only a single source of data. 18 A Note about Credibility of Evidence Applicants are encouraged to provide credible evidence from multiple sources, both quantitative and qualitative. Applicants also should keep in mind that even though their evidence might not have been developed from a research process, they must still consider the same issues that ensure confidence in research results in identifying credible evidence to support claims of effectiveness. For example, anecdotal evidence lacks credibility unless it is obtained through a systematic process. Threats to validity include lack of a representative sample and response bias due to a lack of anonymity. Figure 2 provides examples of levels of credibility for various data sources. Reviewers will use Figure 2 to enhance their understanding of the scope of information that applicants might provide. Applicants may find Figure 2 helpful in making decisions about which evidence to submit with the application. 19 Figure 2. Examples of Credibility Across Sources Teacher Preservice Preparation Teachers’ Faculty Work Most Credible systematic Preservice Students & Program Graduates Data collected on all participants in all stages of the program Supervisory or Mentor Teacher Results from valid and reliable observation instrument rated by a trained observer K–12 Student Results Performance assessment results or results from a test developed by an “expert” (experienced teacher or commercial test developer), reflecting student gains on a topic that beginning teacher taught, evidence of preparedness for next or related courses, increases in higher-level course enrollment Beginning teacher-created test, given pre and post instruction, reflecting student learning, teacher or student reflection logs or journals indicating increased student engagement in learning Beginning teacher rated samples of K–12 student work demonstrating learning Faculty sampling and rating of K–12 student work in preservice teachers’ classes using a valid and reliable rating tool Ratings by an external panel, with no knowledge about the identity of the preservice teacher’s institution, of a systematic portfolio sample reflecting learning Faculty systematic observation of preservice teachers’ classroom instruction Presentation of K–12 student work organized as evidence of preservice teachers’ influence on K–12 student learning Data collected on all graduates. Data include teacher reflections Systematic ratings on a random sample, including mentor teacher assessment of K–12 student learning Faculty review and rating of preservice teachers’ practice teaching Portfolios with section specified to address K–12 student learning Data collected on selected participants in various stages of the program Systematic ratings on practicing teachers or beginning teachers, indicating whether K–12 students have learned Informal reports indicating that K–12 students learned from practicing teacher or beginning teacher Least Narrative Credible report of preservice teachers’ learning in a teacher preparation class Preservice teachers’ portfolios-no systematized ratings Data collected from selected graduates Selected K–12 student comments about the beginning teacher from an evaluation page 20 Applicant Contact Information 1. Program Information Name of Program: Address: Phone: 2. FAX: Contact Information (person in the program with whom U.S. Department of Education staff or site team can communicate about the application throughout the review process) Name: Title/Position: Address: Email: Summer contact information if different from above: 3. Names and positions of the team involved in the completion of this application: 21